Method and apparatus for judging direction of blur and computer-readable recording medium storing a program therefor

ABSTRACT

Without use of special hardware such as an angular rate sensor, a direction of blur can be judged with high accuracy. A parameter acquisition unit obtains weighting parameters for principal components representing directions of blur in a predetermined structure in an image by fitting to the structure a mathematical model generated by a method of AAM using a plurality of sample images representing the structure in different conditions of blur. A blur direction judgment unit judges the direction of blur based on the weighting parameters, and a blur width acquisition unit finds a width of blur based on an edge component found by an edge detection unit in the direction perpendicular to the direction of blur. A blur correction unit corrects the blur in the image based on the direction and the width of blur.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method of judging a direction of blurrepresented in an input image and an apparatus therefor. The presentinvention also relates to a computer-readable recording medium storing aprogram that causes a computer to execute the method.

2. Description of the Related Art

As a result of reduction in size and weight of imaging devices such asdigital cameras and camera phones, camera shake tends to occur due toimaging device movement at the time of shuttering. In addition, progressis being made in increase in the number of pixels and in installation ofa zooming function as standard equipment. Therefore, camera shake tendsto become conspicuous in photographed images.

As a method of detecting such camera shake has been known a method usingan angular rate sensor installed in an imaging device (see U.S. PatentApplication Publication No. 20030002746).

However, the method described in U.S. Patent Application Publication No.20030002746 needs special hardware such as an angular rate sensor, andinstallation of such hardware in a small imaging device such as a cameraphone is difficult.

Furthermore, no method has been disclosed for detecting camera shakebased only on a photographed image in the case where no information isavailable on camera shake detected by such hardware.

SUMMARY OF THE INVENTION

The present invention has been conceived based on consideration of theabove circumstances. An object of the present invention is therefore toprovide a method enabling judgment of a direction of blur such as camerashake with high accuracy and without special hardware such as an angularrate sensor, an apparatus for carrying out the judgment, and acomputer-readable recording medium storing a program therefor.

A blur direction judgment method of the present invention comprises thesteps of:

obtaining one or more weighting parameters for one or more statisticalcharacteristic quantities representing a direction of blur in apredetermined structure in an input image by fitting a modelrepresenting the structure to the structure in the input image, themodel having been obtained by carrying out predetermined statisticalprocessing on a plurality of images representing the predeterminedstructure in different conditions of blur, and the model representingthe structure by one or more statistical characteristic quantitiesincluding the statistical characteristic quantity or quantitiesrepresenting the direction of blur and by one or more weightingparameters for weighting the statistical characteristic quantity orquantities according to an individual characteristic of the structure;and

judging the direction of blur represented in the input image accordingto a value or values of the weighting parameter or parameters havingbeen obtained.

A blur direction judgment apparatus of the present invention is anapparatus for carrying out the blur direction judgment method describedabove. More specifically, the blur direction judgment apparatus of thepresent invention comprises:

parameter acquisition means for obtaining one or more weightingparameters for one or more statistical characteristic quantitiesrepresenting a direction of blur in a predetermined structure in aninput image by fitting a model representing the structure to thestructure in the input image, the model having been obtained by carryingout predetermined statistical processing on a plurality of imagesrepresenting the predetermined structure in different conditions ofblur, and the model representing the structure by one or morestatistical characteristic quantities including the statisticalcharacteristic quantity or quantities representing the direction of blurand by one or more weighting parameters for weighting the statisticalcharacteristic quantity or quantities according to an individualcharacteristic of the structure; and

blur direction judgment means for judging the direction of blurrepresented in the input image according to a value or values of theweighting parameter or parameters having been obtained by the parameteracquisition means.

A computer-readable recording medium of the present invention stores aprogram that causes a computer to execute the blur direction judgmentmethod described above (that is, a program that causes a computer tofunction as the means described above).

Hereinafter, the blur direction judgment apparatus, the blur directionjudgment method, and the blur direction judgment program of the presentinvention are described in detail.

As a method of generating the model representing the predeterminedstructure in the present invention, a method of AAM (Active AppearanceModel) can be used. An AAM is one of approaches in interpretation of thecontent of an image by using a model. For example, in the case where ahuman face is a target of interpretation, a mathematical model of humanface is generated by carrying out principal component analysis on faceshapes in a plurality of images to be learned and on information ofluminance after normalization of the shapes. A face in a new input imageis then represented by principal components in the mathematical modeland corresponding weighting parameters, for face image reconstruction(T. F. Cootes et al., “Active Appearance Models”, Proc. EuropeanConference on Computer Vision, vol. 2, pp. 484-498, Springer, 1998,Germany; hereinafter referred to as Document 1).

The blur may be camera shake or movement of a subject.

The conditions of blur represent at least the direction of blur. Inaddition, the conditions may include a condition representing themagnitude of blur.

It is preferable for the predetermined structure to be suitable formodeling. In other words, variations in shape and luminance of thepredetermined structure in images thereof preferably fall within apredetermined range. Especially, it is preferable for the predeterminedstructure to generate the statistical characteristic quantity orquantities contributing more to the shape and luminance thereof throughthe statistical processing thereon. Furthermore, it is preferable forthe predetermined structure to have a high probability of being a mainpart of image. More specifically, the predetermined structure can be ahuman face.

The plurality of images representing the predetermined structure indifferent conditions of blur may be images obtained by actuallyphotographing the predetermined structure in the different conditions ofblur. Alternatively, the images may be generated through simulationbased on an image of the structure photographed in a specific conditionof blur.

It is preferable for the predetermined statistical processing to bedimension reduction processing that can represent the predeterminedstructure by the statistical characteristic quantity or quantities offewer dimensions than the number of pixels representing thepredetermined structure. More specifically, the predeterminedstatistical processing may be multivariate analysis such as principalcomponent analysis. In the case where principal component analysis iscarried out as the predetermined statistical processing, the statisticalcharacteristic quantity or quantities refers/refer to principalcomponents obtained through the principal component analysis.

In the case where the predetermined statistical processing is principalcomponent analysis, principal components of higher orders contributemore to the shape and luminance than principal components of lowerorders.

In the statistical characteristic quantity or quantities, at leastinformation based on luminance in the structure needs to be represented,since the blur is represented by distribution of luminance in the image.

The statistical characteristic quantity or quantities representing thedirection of blur may be a single statistical characteristic quantity ora plurality of statistical characteristic quantities. For example, thestatistical characteristic quantity or quantities may be statisticalcharacteristic quantities that vary in respective directions of blur.

The (predetermined) structure in the input image may be detectedautomatically or manually. In addition, the present invention mayfurther comprise the step (or means) for detecting the structure in theinput image. Alternatively, the structure may have been detected in theinput image in the present invention.

A plurality of models in the present invention may be prepared forrespective properties of the predetermined structure. In this case, thesteps (or means) may be added to the present invention for obtaining atleast one property of the structure in the input image and for selectingone of the models according to the at least one obtained property. Theweighting parameter or parameters can be obtained by fitting theselected model to the structure in the input image.

The properties refer to gender, age, and race in the case where thepredetermined structure is human face. The property may be informationfor identifying an individual. In this case, the models for therespective properties refer to models for respective individuals.

As a specific method of obtaining the property may be listed imagerecognition processing having been known (such as image recognitionprocessing described in Japanese Unexamined Patent Publication No. 11(1999) -175724). Alternatively, the property may be inferred or obtainedbased on information such as GPS information accompanying the inputimage.

Fitting the model representing the structure to the structure in theinput image refers to calculation or the like for representing thestructure in the input image by the model. More specifically, in thecase where the method of AAM described above is used, fitting the modelrefers to finding values of weighting parameters for the respectiveprincipal components in the mathematical model.

As a specific method of judging the direction of blur represented in theinput image according to the weighting parameter or parameters havingbeen obtained for the statistical characteristic quantity or quantitiesrepresenting the direction of blur, a relationship is experimentally andstatistically found in advance between the direction of blur and thevalue or values of the weighting parameter or parameters for thestatistical characteristic quantity or quantities representing thedirection of blur. Based on the relationship is then found the directionof blur corresponding to the value or values of the weighting parameteror parameters for the statistical characteristic quantity or quantitiesobtained by fitting the model to the structure in the input image.

A width of blur may also be found in the direction of blur having beenjudged. More specifically, the width may be found based on the value orvalues of the weighting parameter or parameters for the statisticalcharacteristic quantity or quantities representing the direction of blurhaving been judged. In this case, a relationship is experimentally andstatistically found in advance between the width of blur in thedirection thereof and the value or values of the weighting parameter orparameters for the statistical characteristic quantity or quantitiesrepresenting the direction of blur, and the width of blur is found basedon the relationship. Alternatively, a width of an edge perpendicular tothe direction of blur having been judged may be found and used as thewidth of blur. In this case, edges perpendicular to the direction ofblur are found through direction-dependent edge detection processinghaving been known. A mean width of the detected edges is found as thewidth of blur.

Furthermore, the blur represented in the input image may be correctedbased on the direction of blur having been judged and the width of blurhaving been found. At this time, the blur in the structure in the inputimage may be corrected by changing the value or values of the weightingparameter or parameters for the statistical characteristic quantity orquantities representing the direction of blur. In addition, sharpnesscorrection may be carried out on the entire input image for enhancingthe edges in the direction substantially perpendicular to the directionof blur having been judged, at processing strength in accordance withthe width of blur having been found (see Japanese Unexamined PatentPublication No. 2004-070421). Furthermore, blur correction may becarried out on the structure in the input image by changing the value orvalue of the weighting parameter or parameters and on the remaining partof the input image according to the sharpness correction describedabove.

According to the blur direction judgment method, the blur directionjudgment apparatus, and the computer-readable recording medium storingthe blur direction judgment program of the present invention, theweighting parameter or parameters for the statistical characteristicquantity or quantities representing the direction of blur in thestructure in the input image is/are obtained by fitting to the structurein the input image the model representing the predetermined structure byuse of the statistical characteristic quantity or quantities includingthe statistical characteristic quantity or quantities representing thedirection of blur and the weighting parameter or parameters therefor.Based on the value or values of the weighting parameter or parametershaving been obtained, the direction of blur represented in the inputimage is judged. Therefore, without use of special hardware such as anangular rate sensor, blur such as camera shake can be detected with highaccuracy, and the direction of blur can be judged in the presentinvention in a photographed image even in the case where no informationon blur is detected at the time of photography.

In the case where the step (or the means) for detecting the structure inthe input image is added, automatic detection of the structure can becarried out, leading to improvement in operability.

In the case where the plurality of models are prepared for therespective properties of the predetermined structure in the presentinvention while the steps (or the means) are added for obtaining theproperty of the structure in the input image and for selecting one ofthe models in accordance with the at least one obtained property, if theweighting parameter or parameters is/are obtained by fitting theselected model to the structure in the input image, the structure in theinput image can be fit by the model that is more suitable. Therefore,processing accuracy is improved.

In the case where the steps (means) are added for finding the width ofblur in the direction of blur having been judged and for correcting theblur in the input image based on the direction of blur and the width ofblur having been found, the blur can be corrected based on a result ofhigh-accuracy blur detection, which leads to generation of a preferableimage.

In the case where the width of blur is found based on the value orvalues of the weighting parameter or parameters for the statisticalcharacteristic quantity or quantities representing the direction ofblur, the width of blur can be found without image analysis processingsuch as edge detection, which enables simpler configuration andimprovement in processing efficiency.

Likewise, in the case where the blur in the structure in the input imageis corrected by changing the value or values of the weighting parameteror parameters for the statistical characteristic quantity or quantitiesrepresenting the direction of blur, the blur can be corrected withoutother image processing such as sharpness correction, which enablessimpler configuration and improvement in processing efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows hardware configuration of a digital photograph printer inan embodiment of the present invention;

FIG. 2 is a block diagram showing functions and a flow of processing inthe digital photograph printer in the embodiment and in a digital camerain another embodiment of the present invention;

FIGS. 3A and 3B show examples of screens displayed on a display of thedigital photograph printer and the digital camera in the embodiments;

FIG. 4 is a block diagram showing details of blur direction judgingprocess and blur correcting process in one aspect of the presentinvention;

FIG. 5 is a flow chart showing a procedure for generating a mathematicalmodel of face image in the present invention;

FIG. 6 shows an example of how feature points are set in a face;

FIG. 7 shows how a face shape changes with change in values of weightcoefficients for eigenvectors of principal components obtained throughprincipal component analysis on the face shape;

FIG. 8 shows luminance in mean face shapes converted from face shapes insample images;

FIG. 9 shows how luminance in a face change with change in values ofweight coefficients for eigenvectors of principal components obtained byprincipal component analysis on the luminance in the face;

FIG. 10 is a block diagram showing details of blur direction judgingprocess and blur correcting process in another aspect of the presentinvention;

FIG. 11 shows a structure of a reference table used in the blurcorrecting process and an example of values therein;

FIG. 12 is a block diagram showing an advanced aspect of the blurcorrecting process in the present invention; and

FIG. 13 shows the configuration of the digital camera in the embodimentof the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, embodiments of the present invention are described withreference to the accompanying drawings.

FIG. 1 shows the hardware configuration of a digital photograph printerin an embodiment of the present invention. As shown in FIG. 1, thedigital photograph printer comprises a film scanner 51, a flat headscanner 52, a media drive 53, a network adapter 54, a display 55, akeyboard 56, a mouse 57, a hard disc 58, and a photographic print outputmachine 59, all of which are connected to an arithmetic and control unit50.

In cooperation with a CPU, a main storage, and various input/outputinterfaces, the arithmetic and control unit 50 controls a processingflow regarding an image, such as input, correction, manipulation, andoutput thereof, by executing a program installed from a recording mediumsuch as a CD-ROM. In addition, the arithmetic and control unit 50carries out image processing calculation for image correction andmanipulation. A blur direction judging process and a blur correctingprocess of the present invention is also carried out by the arithmeticand control unit 50.

The film scanner 51 photoelectrically reads an APS negative film or a135-mm negative film developed by a film developer (not shown) forobtaining digital image data P0 representing a photograph image recordedon the negative film.

The flat head scanner 52 photoelectrically reads a photograph imagerepresented in the form of hard copy such as an L-size print, forobtaining digital image data P0.

The media drive 53 obtains digital image data P0 representing aphotograph image recorded in a recording medium such as a memory card, aCD, or a DVD. The media drive 53 can also write image data P2 to beoutput therein. The memory card stores image data representing an imagephotographed by a digital camera, for example. The CD or the DVD storesdata of an image read by the film scanner regarding a previous printorder, for example.

The network adapter 54 obtains image data P0 from an order receptionmachine (not shown) in a network photograph service system having beenknown. The image data P0 are image data used for a photograph printorder placed by a user, and sent from a personal computer of the uservia the Internet or via a photograph order reception machine installedin a photo laboratory.

The display 55 displays an operation screen for input, correction,manipulation, and output of an image carried out by the digitalphotograph printer. A menu for selecting the content of operation and animage to be processed are also displayed thereon, for example. Thekeyboard 56 and the mouse 57 are used for inputting a processinginstruction.

The hard disc 58 stores a program for controlling the digital photographprinter. In the hard disc 58 are also stored temporarily the image dataP0 obtained by the film scanner 51, the flat head scanner 52, the mediadrive 53, and the network adapter 54, in addition to image data P1having been subjected to image correction (hereinafter referred to asthe corrected image data P1) and the image data P2 having been subjectedto image manipulation (the image data to be output).

The photograph print output machine 59 carries out laser scanningexposure of photographic printing paper, image development thereon, anddrying thereof, based on the image data P2 representing the image to beoutput. The photograph print output machine 59 also prints printinginformation on the backside of the paper, cuts the paper for each print,and sorts the paper for each order. The manner of printing may be alaser exposure thermal development dye transfer process or the like.

FIG. 2 is a block diagram showing functions of the digital photographprinter and the flow of processing carried out therein. As shown in FIG.2, the digital photograph printer comprises image input means 1, imagecorrection means 2, image manipulation means 3, and image output means 4in terms of the functions. The image input means 1 inputs the image dataP0 of an image to be printed. The image correction means 2 uses theimage data P0 as input, and carries out automatic image qualitycorrection of the image represented by the image data P0 (hereinafter,image data and an image represented by the image data are represented bythe same reference code) through image processing according to apredetermined image processing condition. The image manipulation means 3uses the corrected image data P1 having been subjected to the automaticcorrection as input, and carries out image processing according to aninstruction from an operator. The image output means 4 uses theprocessed image data P2 as input, and outputs a photographic print oroutputs the processed image data P2 in a recording medium.

The image correction means 2 carries out processing such as whitebalance adjustment, contrast correction, sharpness correction, and noisereduction and removal. The image manipulation means 3 carries out manualcorrection on a result of the processing carried out by the imagecorrection means 2. In addition, the image manipulation means 3 carriesout image manipulation such as trimming, scaling, conversion to sepiaimage, conversion to monochrome image, and compositing with anornamental frame, in addition to the blur direction judging process andthe blur correcting process of the present invention.

Operation of the digital photograph printer and the flow of theprocessing therein are described next.

The image input means 1 firstly inputs the image data P0. In the casewhere an image recorded on a developed film is printed, the operatorsets the film on the film scanner 51. In the case where image datastored in a recording medium such as a memory card are printed, theoperator sets the recording medium in the media drive 53. A screen forselecting a source of input of the image data is displayed on thedisplay 55, and the operator carries out the selection by using thekeyboard 56 or the mouse 57. In the case where “film” has been selectedas the source of input, the film scanner 51 photoelectrically reads thefilm set thereon, and carries out digital conversion. The image data P0generated in this manner are then sent to the arithmetic and controlunit 50. In the case where “hard copy” such as a photographic print hasbeen selected, the flat head scanner 52 photoelectrically reads the hardcopy set thereon, and carries out digital conversion. The image data P0generated in this manner are then sent to the arithmetic and controlunit 50. In the case where “recording medium” such as a memory card hasbeen selected, the arithmetic and control unit 50 reads the image dataP0 stored in the recording medium such as a memory card set in the mediadrive 53. In the case where an order has been placed in a networkphotograph service system or by a photograph order reception machine ina store, the arithmetic and control unit 50 receives the image data P0via the network adapter 54. The image data P0 obtained in this mannerare temporarily stored in the hard disc 58.

The image correction means 2 then carries out the automatic imagequality correction on the image represented by the image data P0. Morespecifically, publicly known processing such as white balanceadjustment, contrast correction, sharpness correction, and noisereduction and removal is carried out based on a setup condition set onthe printer in advance, according to an image processing programexecuted by the arithmetic and control unit 50. The corrected image dataP1 are output to be stored in a memory of the arithmetic and controlunit 50. Alternatively, the corrected image data P1 may be storedtemporarily in the hard disc 58.

The image manipulation means 3 thereafter generates a thumbnail image ofthe corrected image P1, and causes the display 55 to display thethumbnail image. FIG. 3A shows an example of a screen displayed on thedisplay 55. The operator confirms displayed thumbnail images, andselects any one of the thumbnail images that needs manual image-qualitycorrection or order processing for image manipulation while using thekeyboard 56 or the mouse 57. In FIG. 3A, the image in the upper leftcorner (DSCF0001) is selected. As shown in FIG. 3B as an example, theselected thumbnail image is enlarged and displayed on the display 55,and buttons are displayed for selecting the content of manual correctionand manipulation on the image. The operator selects a desired one of thebuttons by using the keyboard 56 or the mouse 57, and carries outdetailed setting of the selected content if necessary. The imagemanipulation means 3 carries out the image processing according to theselected content, and outputs the processed image data P2. The imagedata P2 are stored in the memory of the arithmetic and control unit 50or stored temporarily in the hard disc 58. The program executed by thearithmetic and control unit 50 controls image display on the display 55,reception of input from the keyboard 56 or the mouse 57, and imageprocessing such as manual correction and manipulation carried out by theimage manipulation means 3.

The image output means 4 finally outputs the image P2. The arithmeticand control unit 50 causes the display 55 to display a screen for imagedestination selection, and the operator selects a desired one ofdestinations by using the keyboard 56 or the mouse 57. The arithmeticand control unit 50 sends the image data P2 to the selected destination.In the case where a photographic print is generated, the image data P2are sent to the photographic print output machine 59 by which the imagedata P2 are output as a photographic print. In the case where the imagedata P2 are recorded in a recording medium such as a CD, the image dataP2 are written in the CD or the like set in the media drive 53.

Below is described the blur direction judging process of the presentinvention followed by the blur correcting process carried out in thecase where “Blur Correction” is selected in the screen shown in FIG. 3B.FIG. 4 is a block diagram showing details of the blur direction judgingprocess and the blur correcting process. As shown in FIG. 4, the blurdirection judging process and the blur correcting process is carried outby a face detection unit 31, a parameter acquisition unit 32, a blurdirection judgment unit 33, an edge detection unit 34, a blur widthacquisition unit 35, and a blur correction unit 36. The face detectionunit 31 detects a face region P1 f in the image P1. The parameteracquisition unit 32 fits to the detected face region P1 f a mathematicalmodel M generated by a method of AAM (see aforementioned Document 1)based on a plurality of sample images representing human faces indifferent conditions of blur, and obtains weight coefficients λ_(d1),λ_(d2), λ_(d3), . . . , λ_(dJ), for principal components representingdirections of blur in the face region P1 f. The blur direction judgmentunit 33 judges a direction D of blur based on the weight coefficientshaving been obtained. The edge detection unit 34 detects edge componentsE perpendicular to the direction D in the image P1. The blur widthacquisition unit 35 finds a width W of blur based on the edge componentE. The blur correction unit 36 corrects the blur in the input image P1based on the direction D and the width W of blur. The processingdescribed above is controlled by the program installed in the arithmeticand control unit 50.

The mathematical model M is generated according to a flow chart shown inFIG. 5, and installed in advance together with the programs describedabove. Hereinafter, how the mathematical model M is generated isdescribed.

For each of the sample images representing human faces in differentdirections and magnitudes of blur, feature points are set therein asshown in FIG. 6 for representing face shape (Step #1). In this case, thenumber of the feature points is 122. However, only 60 points are shownin FIG. 6 for simplification. Which part of face is represented by whichof the feature points is predetermined, such as the left corner of theleft eye represented by the first feature point and the center betweenthe eyebrows represented by the 38^(th) feature point. Each of thefeature points maybe set manually or automatically according torecognition processing. Alternatively, the feature points maybe setautomatically and later corrected manually upon necessity.

Based on the feature points set in each of the sample images, mean faceshape is calculated (Step #2). More specifically, mean values ofcoordinates of the feature points representing the same part are foundamong the sample images.

Principal component analysis is then carried out based on thecoordinates of the mean face shape and the feature points representingthe face shape in each of the sample images (Step #3). As a result, anyface shape can be approximated by Equation (1) below: $\begin{matrix}{S = {S_{0} + {\sum\limits_{i = 1}^{n}{p_{i}b_{i}}}}} & (1)\end{matrix}$

S and S₀ are shape vectors represented respectively by simply listingthe coordinates of the feature points (x₁, Y₁, . . . , X₁₂₂, Y₁₂₂) inthe face shape and in the mean face shape, while p_(i) and b_(i) are aneigenvector representing the i^(th) principal component for the faceshape obtained by the principal component analysis and a weightcoefficient therefor, respectively. FIG. 7 shows how face shape changeswith change in values of the weight coefficients b₁ and b₂ for theeigenvectors p_(i) and P₂ as the highest and second-highest orderprincipal components obtained by the principal component analysis. Thechange ranges from −3sd to +3sd where sd refers to standard deviation ofeach of the weight coefficients b₁ and b₂ in the case where the faceshape in each of the sample images is represented by Equation (1). Theface shape in the middle of 3 faces for each of the componentsrepresents the face shape in the case where the value of thecorresponding weight coefficient is the mean value. In this example, acomponent contributing to face outline has been extracted as the firstprincipal component through the principal component analysis. Bychanging the weight coefficient b₁, the face shape changes from anelongated shape (corresponding to −3sd) to a round shape (correspondingto +3sd). Likewise, a component contributing to how much the mouth isopen and to length of chin has been extracted as the second principalcomponent. By changing the weight coefficient b₂, the face changes froma state of open mouth and long chin (corresponding to −3sd) to a stateof closed mouth and short chin (corresponding to +3sd). The smaller thevalue of i, the better the component explains the shape. In other words,the i_(th) component contributes more to the face shape as the value ofi becomes smaller.

Each of the sample images is then subjected to conversion (warping) intothe mean face shape obtained at Step #2 (Step #4). More specifically,shift values are found between each of the sample images and the meanface shape, for the respective feature points. In order to warp pixelsin each of the sample images to the mean face shape, shift values to themean face shape are calculated for the respective pixels in each of thesample images according to 2-dimensional 5-degree polynomials (2) to (5)using the shift values having been found: $\begin{matrix}{x^{\prime} = {x + {\Delta\quad x}}} & (2) \\{y^{\prime} = {y + {\Delta\quad y}}} & (3) \\{{\Delta\quad x} = {\sum\limits_{i = 0}^{n}{\sum\limits_{j = 0}^{n - i}{a_{ij} \cdot x^{i} \cdot y^{j}}}}} & (4) \\{{\Delta\quad x} = {\sum\limits_{i = 0}^{n}{\sum\limits_{j = 0}^{n - i}{b_{ij} \cdot x^{i} \cdot y^{j}}}}} & (5)\end{matrix}$

In Equations (2) to (5) above, x and y denote the coordinates of each ofthe feature points in each of the sample images while x′ and y′ arecoordinates in the mean face shape to which x and y are warped. Theshift values to the mean shape are represented by Δx and Δy with n beingthe number of dimensions while a_(ij) and b_(ij) are coefficients. Thecoefficients for polynomial approximation can be found by using a leastsquare method. In the case that the coordinates of a feature pointbecome non-integer values (that is, values including decimals), pixelvalues of four pixels (having integer coordinates) that surround thecoordinates after warping are found through linear approximation of thepixel values. More specifically, for 4 pixels surrounding thecoordinates of the non-integer values generated by warping, the pixelvalues for each of the 4 pixels are determined in proportion to adistance thereto from the coordinates generated by warping. FIG. 8 showshow the face shape of each of 3 sample images is changed to the meanface shape.

Thereafter, principal component analysis is carried out, using asvariables the luminance of each of the pixels in each of the sampleimages after the change to the mean face shape (Step #5). As a result,the luminance in the mean face shape converted from any arbitrary faceimage can be approximated by Equation (6) below: $\begin{matrix}{A = {A_{0} + {\sum\limits_{i = 1}^{m}{q_{i}\lambda_{i}}}}} & (6)\end{matrix}$

In Equation (6), A denotes a luminance vector (a₁, . . . , a_(m))represented by listing the luminance at each of the pixels in the meanface shape (where a represents the luminance while 1 to m refer tosubscripts for identifying the respective pixels with m being the totalnumber of pixels in the mean face shape). A₀ is a mean face luminancevector represented by listing mean values of the luminance at each ofthe pixels in the mean face shape while q_(i) and λ_(i) refer to aneigenvector representing the i^(th) principal component for theluminance in the face obtained by the principal component analysis and aweight coefficient therefor, respectively. The smaller the value of iis, the better the component explains the luminance. In other words, thecomponent contributes more to the luminance as the value of i becomessmaller.

FIG. 9 shows how the luminance values of faces change with change invalues of the weight coefficients λ_(i1) and λ_(i2) for the eigenvectorsq_(i1) and q_(i2) representing the i₁ ^(th) and i₂ ^(th) principalcomponents obtained through the principal component analysis. The changein the weight coefficients ranges from −3sd to +3sd where sd refers tostandard deviation of each of the values of the weight coefficientsλ_(i1), and λ_(i2) in the case where the luminance in each of the facesample images are represented by Equation (6) above. For each of theprincipal components, the face in the middle of the 3 images correspondsto the case where the corresponding weight coefficient λ_(i1), or λ_(i2)is the mean value. In the examples shown in FIG. 9, a componentcontributing to presence or absence of beard has been extracted as thei₁ ^(th) principal component through the principal component analysis.By changing the weight coefficient λ_(i1), the face changes from theface with dense beard (corresponding to −3sd) to the face with no beard(corresponding to +3sd). Since the face images having differentdirections and magnitudes of blur are used as the sample images in thisembodiment, a component contributing to blur has been extracted as thei₂ ^(th) principal component through the principal component analysis.By changing the weight coefficient λ_(i2), the magnitude of blur in theright-left direction changes. Likewise, components contributing to blurin other directions are extracted as other principal components. Howeach of the principal components contributes to what factor isdetermined through interpretation. The directions of blur can berepresented by a plurality of principal components as in the aboveexample or by a single principal component.

Through the processing from Step #1 to #5 described above, themathematical model M is generated. In other words, the mathematicalmodel M is represented by the eigenvectors p_(i) representing the faceshape and the eigenvectors q_(i) representing the face luminance in themean face shape, and the number of the eigenvectors is far smaller forp_(i) and for q_(i) than the number of pixels forming the face image. Inother words, the mathematical model M has been compressed in terms ofdimension. In the example described in Document 1, 122 feature pointsare set for a face image of approximately 10, 000 pixels, and amathematical model of face image represented by 23 eigenvectors for faceshape and 114 eigenvectors for face luminance has been generated throughthe processing described above. By changing the weight coefficients forthe respective eigenvectors, 90% of variations or more in face shape andluminance can be expressed.

A flow of the blur direction judging process and the blur correctingprocess based on the method of AAM using the mathematical model M isdescribed below, with reference to FIG. 4.

The face detection unit 31 reads the image data P1, and detects the faceregion P1 f in the image P1. More specifically, a first characteristicquantity representing a direction of a gradient vector showing adirection and a magnitude of an edge at each of the pixels in the imageP1 is firstly input to a plurality of first detectors (which will bedescribed later) for judgment as to whether a face candidate regionexists in the image P1, as has been described in Japanese UnexaminedPatent Publication No. 2005-108195 (hereinafter referred to as Reference2). In the case where a face candidate region exists, the region isextracted, and the magnitude of the gradient vector is normalized ateach of the pixels in the face candidate region. A second characteristicquantity representing the direction and the magnitude of the normalizedgradient vector is then input to second detectors (which will bedescribed later) for judgment as to whether the extracted face candidateregion is a true face region. In the case where the region is a trueface region, the region is detected as the face region P1 f. Thefirst/second detectors have been obtained through training using amethod of machine learning such as AdaBoost that uses as input thefirst/second characteristic quantity calculated for face sample imagesand non-face sample images.

As a method of detection of the face region P1 f may be used variousknown methods such as a method using a correlation score between aneigen-face representation and an image as has been described inPublished Japanese Translation of a PCT Application No. 2004-527863(hereinafter referred to as Reference 3). Alternatively, the face regioncan be detected by using a knowledge base, characteristics extraction,skin-color detection, template matching, graph matching, and astatistical method (such as a method using a neural network, SVM, andHMM), for example. Furthermore, the face region P1 f may be specifiedmanually by use of the keyboard 56 and the mouse 57 when the image P1 isdisplayed on the display 55. Alternatively, a result of automaticdetection of the face region may be corrected manually.

The parameter acquisition unit 32 fits the mathematical model M to theface region P1 f. More specifically, the parameter acquisition unit 32reconstructs the image according to Equations (1) and (6) describedabove while sequentially changing the values of the weight coefficientsb_(i) and λ_(i) for the eigenvectors p_(i) and q_(i) corresponding tothe principal components in order of higher order in Equations (1) and(6). The parameter acquisition unit 32 then finds the values of theweight coefficients b_(i) and λ_(i) causing a difference between thereconstructed image and the face region P1 f to become minimal (seeReference 3 for details). Among the weight coefficients having beenfound, the weight coefficients corresponding to the principal componentsrepresenting the directions of blur are the weight coefficients λ_(d1),λ_(d2), λ_(d3), . . . , λ_(dJ). It is preferable for the values of theweight coefficients b_(i) and λ_(i) to range only from −3sd to +3sd, forexample, where sd refers to the standard deviation in each ofdistributions of b_(i) and λ_(i) when the sample images used at the timeof generation of the model are represented by Equations (1) and (6). Inthe case where the values do not fall within the range, it is preferablefor the weight coefficients to take the mean values in thedistributions. In this manner, erroneous application of the model can beavoided.

The blur direction judgment unit 33 judges the direction D of blur basedon the weight coefficients λ_(d1), λ_(d2), λ_(d3), . . . , λ_(dJ) havingbeen obtained. More specifically, a direction vector representing thedirection of blur is found in the face region P1 f by adding unitdirection vectors representing the respective directions of blurweighted by the absolute values of the weight coefficients correspondingthereto, and the direction represented by the direction vector is usedas the direction D of the blur. Alternatively, a reference table may befound experimentally and statistically in advance for defining arelationship between a combination of the values of the weightcoefficients λ_(d1), λ_(d2), λ_(d3), . . . , λ_(dJ) and the direction Dof the blur. The direction D of blur may be found with reference to thereference table according to the values of the weight coefficientsλ_(d1), λ_(d2), λ_(d3), . . . , λ_(dJ) having been obtained.

The edge detection unit 34 detects the edge components E in thedirection perpendicular to the direction D through direction-dependentedge detection processing according to an operator E_(i) (where i=1,2, .. . denotes an index for identifying each of the directions of edge)that detects an edge perpendicular to the direction D by using spatialfirst derivative or the like.

The blur width acquisition unit 35 finds a mean value of widths of theedge components E as the width W.

The blur correction unit 36 refers to a reference table Tw based on thewidth W of blur, and obtains an enhancement factor a (where j (=1,2, . .. ) refers to an index for identifying the enhancement factor)representing a degree of edge enhancement according to the magnitude ofthe width W. The reference table T_(w), defines a relationship betweenthe width W of blur and the enhancement factor α_(j), based onexperimental and statistical data.

The blur correction unit 36 further carries out selective sharpening forenhancing the edges in the direction perpendicular to the direction ofblur in the image data P1, based on the enhancement factor αj and theoperator E_(i) once used by the edge detection unit 34. The blurcorrection unit 36 then outputs the image data P2. More specifically,the blur correction unit 36 carries out the conversion processingrepresented by the following equation:P2(x,y)=P1(x,y)+α_(j) ·E _(i) ·L(x,y)  (7)where P2 (x,y) and P1 (x,y) respectively denote the output image and theinput image while L (x,y) refers to a Laplacian image of the input imageP1.

As has been described above, according to the blur direction judgingprocess and the blur correcting process in the embodiment of the presentinvention, the parameter acquisition unit 32 obtains the weightcoefficients λ_(d1), λ_(d2), λ_(d3), . . . , λ_(dJ) corresponding to theprincipal components representing the directions of blur in the faceregion P1 f detected by the face detection unit 31 in the image P1 byfitting to the face region P1 f the mathematical model M generatedaccording to the method of AAM using the sample images representinghuman faces in different conditions of blur. The blur direction judgmentunit 33 then judges the direction D of blur based on the weightcoefficients λ_(d1), λ_(d2), λ_(d3), . . . , λ_(dJ) and the blur widthacquisition unit 35 finds the width W of blur based on the edgecomponents E detected by the edge detection unit 34 in the directionperpendicular to the direction D. The blur correction unit 36 thencorrects the blur in the input image P1 based on the direction D and thewidth W of blur. Therefore, without special hardware such as an angularrate sensor, the detection and correction of blur can be carried outwith high accuracy, and the image can be obtained in a more preferablestate.

In the embodiment described above, the edge detection unit 34 may findthe operator E_(i) based on the weight coefficients λ_(d1), λ_(d2),λ_(d3), . . . , λ_(dJ) by referring to a reference table foundexperimentally and statistically in advance for defining a relationshipbetween the operator E_(i) and the combination of the values of theweight coefficients λ_(d1), λ_(d2), λ_(d3), . . . , λ_(dJ) withoutexplicit judgment of the direction D of blur by the blur directionjudgment unit 33. In this case, since the operator E_(i) that detectsthe edge perpendicular to the direction of blur is found with referenceto the reference table, the direction D of blur is implicitly judged.

In the embodiment described above, the edge detection unit 34 mayextract the edges in the direction perpendicular to the direction D ofblur, among the edges in the directions and the magnitudes calculated bythe face detection unit 31 at the respective pixels in the image P1 forfinding the first characteristic quantity. In this manner, redundantedge detection is avoided, which leads to improvement in processingefficiency.

In the embodiment described above, the weight coefficients λ_(d1),λ_(d2), λ_(d3), . . . , λ_(dJ) contribute only to the direction D ofblur. However, in the case where the weight coefficients λ_(d1), λ_(d2),λ_(d3), . . . , λ_(dJ) also contribute to the width of blur in therespective directions corresponding thereto (such as the case where thewidth becomes larger as the absolute value of each of the weightcoefficients λ_(d1), λ_(d2), λ_(d3), . . . , λ_(dJ) becomes larger), thewidth W of blur can be found directly from the weight coefficientsλ_(d1), λ_(d2), λ_(d3), . . . , λ_(dJ) without the processing by theedge detection unit 34. In this manner, the configuration becomessimpler and the processing becomes more efficient.

More specifically, a block diagram in FIG. 10 shows the configuration inthis case. As shown in FIG. 10, a blur correction unit 33′ obtains theoperator Ei for detecting the edge in the direction perpendicular to thedirection of blur and the enhancement factor α_(j) representing thedegree of edge enhancement according to the magnitude of blur, withreference to a reference table T according to the weight coefficientsλ_(d1), λ_(d2), λ_(d3), . . . , λ_(dJ) obtained by the parameteracquisition unit 32. FIG. 11 shows a structure of the reference table Tand an example of values therein. As shown in FIG. 11, the enhancementfactor α_(j) is related to the operator E_(i) for each of the values ofthe weight coefficients λ_(d1), λ_(d2), λ_(d3), . . . , λ_(dJ). Thisrelationship is found experimentally and statistically in advance. Sincethe blur correction unit 33′ finds the operator E_(i) for detecting theedge perpendicular to the direction of blur with reference to thereference table, the blur correction unit 33′ implicitly detects thedirection D of blur.

Before the reference table is referred, a combined parameter λ_(D) maybe found as a linear combination of the weight coefficients as shown byEquation (8) below wherein λ_(i) is a coefficient representing a rate ofcontribution of the i^(th) principal component corresponding to theweight coefficient λ^(i) to the direction and the width of blur:$\begin{matrix}{\lambda_{D} = {\sum\limits_{i = 1}^{J}{\beta_{i}\lambda_{i}}}} & (8)\end{matrix}$

In this case, the reference table T needs to relate only the combinedparameter λ_(D) the operator E_(i), and the enhancement factor α_(i).Therefore, the content of setting is simplified.

In addition, a function using the weight coefficients λ_(d1), λ_(d2),λ_(d3), . . . , λ_(dJ) as input and the operator E_(i) and theenhancement factor α_(i) as output may be defined so that the operatorEi and the enhancement factor α_(i) can be found based on the function.

In the embodiment described above, the known selective sharpening isused as the blur correcting process to be carried out on the entireimage P1. However, the blur correction may be carried out on the faceregion P1 f by changing the values of the weight coefficients λ_(d1),λ_(d2), λ_(d3), . . . , λ_(dJ) to values corresponding to a state withno blur (such as 0).

In the embodiment described above, the mathematical model M is unique.However, a plurality of mathematical models Mi (i=1, 2, . . . ) may begenerated for respective properties such as race, age, and gender, forexample. FIG. 12 is a block diagram showing details of blur directionjudging process and the blur correcting process in this case. As shownin FIG. 12, a property acquisition unit 37 and a model selection unit 38are added, which is different from the embodiment shown in FIG. 4. Theproperty acquisition unit 37 obtains property information A_(K) of asubject in the image P1. The model selection unit 38 selects amathematical model M_(K) generated only from sample images representingsubjects having a property represented by the property informationA_(K).

The mathematical models M_(i) have been generated based on the methoddescribed above (see FIG. 5), only from sample images representingsubjects of the same race, age, and gender, for example. Themathematical models M_(i) are stored by being related to propertyinformation A_(i) representing each of the properties that is commonamong the samples used for the model generation.

The property acquisition unit 37 may obtain the property informationA_(K) by judging the property of the subject through execution of knownrecognition processing (such as processing described in JapaneseUnexamined Patent Publication No. 11 (1999)-175724) on the image P1.Alternatively, the property of the subject maybe recorded at the time ofphotography as accompanying information of the image P1 in a header orthe like so that the recorded information can be obtained. The propertyof the subject may be inferred from accompanying information. In thecase where GPS information representing a photography location isavailable, the country or a region corresponding to the GPS informationcan be identified, for example. Therefore, the race of the subject canbe inferred to some degree. By paying attention to this fact, areference table relating GPS information to information on race may begenerated in advance. By inputting the image P1 obtained by a digitalcamera that obtains the GPS information at the time of photography andrecords the GPS information in a header of the image P1 (such as adigital camera described in Japanese Unexamined Patent Publication No.2004-153428), the GPS information recorded in the header of the imagedata P1 is obtained. The information on race related to the GPSinformation may be inferred as the race of the subject when thereference table is referred to according to the GPS information.

The model selection unit 38 obtains the mathematical model M_(K) relatedto the property information A_(K) obtained by the property acquisitionunit 37, and the parameter acquisition unit 32 fits the mathematicalmodel M_(K) to the face region P1 f in the image P1.

As has been described above, in the case where the mathematical modelsM_(i) corresponding to the properties have been prepared, if the modelselection unit 38 selects the mathematical model M_(K) related to theproperty information A_(K) obtained by the property acquisition unit 37and if the parameter acquisition unit 32 fits the selected mathematicalmodel M_(K) to the face region P1 f, the mathematical model M_(K) doesnot have eigenvectors contributing to variations in face shape andluminance caused by difference in the property represented by theproperty information A_(K). Therefore, the face region P1 f can berepresented only by eigenvectors representing factors determining theface shape and luminance other than the factor representing theproperty. Consequently, processing accuracy improves, and the image canbe obtained in higher quality.

From a viewpoint of improvement in processing accuracy, it is preferablefor the mathematical models for respective properties to be specifiedfurther so that a mathematical model for each individual as a subjectcan be generated. In this case, the image P1 needs to be related toinformation identifying each individual.

In the embodiment described above, the mathematical models are installedin the digital photograph printer in advance. However, from a viewpointof processing accuracy improvement, it is preferable for mathematicalmodels for different human races to be prepared so that which of themathematical models is to be installed can be changed according to acountry or a region to which the digital photograph printer is going tobe shipped.

The function for generating the mathematical model may be installed inthe digital photograph printer. More specifically, a program for causingthe arithmetic and control unit 50 to execute the processing describedby the flow chart in FIG. 5 is installed therein. In addition, a defaultmathematical model may be installed at the time of shipment thereof. Inthis case, the mathematical model may be customized based on imagesinput to the digital photograph printer. Alternatively, a new modeldifferent from the default model may be generated. This is especiallyeffective in the case where the mathematical models for respectiveindividuals are generated.

In the embodiment described above, the individual face image isrepresented by the weight coefficients b_(i) and λ_(i) for the faceshape and the luminance. However, variation in the face shape iscorrelated to variation in the luminance. Therefore, a new appearanceparameter c can be obtained for controlling both the face shape and theluminance as shown by Equations (9) and (10) below, through furtherexecution of principal component analysis on a vector (b_(i), b₂, . . ., b_(i), . . . , λ₁, λ₂, . . . , λ_(i), . . . ) combining the weightcoefficients b_(i) and λ_(i):S=S ₀ +Q _(s) c  (9)A=A ₀ +Q _(A) c  (10)

A difference from the mean face shape can be represented by theappearance parameter c and a vector Q_(s), and a difference from themean luminance can be represented by the appearance parameter c and avector Q_(A).

In the case where this model is used, the parameter acquisition unit 32finds the face luminance in the mean face shape based on Equation (10)above while changing a value of the appearance parameter c. Thereafter,the face image is reconstructed by conversion from the mean face shapeaccording to Equation (9) above, and the value of the appearanceparameter c causing a difference between the reconstructed face imageand the face region P1 f to be minimal is found.

Another embodiment of the present invention can be installation of theblur direction judging process and the blur correcting process in adigital camera. FIG. 13 shows the configuration of such a digitalcamera. As shown in FIG. 13, the digital camera has an imaging unit 71,an A/D conversion unit 72, an image processing unit 73, acompression/decompression unit 74, a flash unit 75, an operation unit76, a media recording unit 77, a display unit 78, a control unit 70, andan internal memory 79. The imaging unit 71 comprises a lens, an iris, ashutter, a CCD, and the like, and photographs a subject. The A/Dconversion unit 72 obtains digital image data P0 by digitizing an analogsignal represented by charges stored in the CCD of the imaging unit 71.The image processing unit 73 carries out various kinds of imageprocessing on image data such as the image data P0. Thecompression/decompression unit 74 carries out compression processing onimage data to be stored in a memory card, and carries out decompressionprocessing on image data read from a memory card in a compressed form.The flash unit 75 comprises a flash and the like, and carries out flashemission. The operation unit 76 comprises various kinds of operationbuttons, and is used for setting a photography condition, an imageprocessing condition, and the like. The media recording unit 77 is usedas an interface with a memory card in which image data are stored. Thedisplay unit 78 comprises a liquid crystal display (hereinafter referredto as the LCD) and the like, and is used for displaying a through image,a photographed image, various setting menus, and the like. The controlunit 70 controls processing carried out by each of the units. Theinternal memory 79 stores a control program, image data, and the like.

The functions of the image input means 1 in FIG. 2 are realized by theimaging unit 71 and the A/D conversion unit 72. Likewise, the functionsof the image correction means 2 are realized by the image processingunit 73 while the functions of the image manipulation means 3 arerealized by the image processing unit 73, the operation unit 76, and thedisplay unit 78. The functions of the image output means 4 are realizedby the media recording unit 77. All of the functions described above arerealized under control of the control unit 70, by using the internalmemory 79 in addition.

Operation of the digital camera and a flow of processing therein aredescribed next.

The imaging unit 71 causes light entering the lens from a subject toform an image on a photoelectric surface of the CCD when a photographerfully presses a shutter button. After photoelectric conversion, theimaging unit 71 outputs an analog image signal, and the A/D conversionunit 72 converts the analog image signal output from the imaging unit 71to a digital image signal. The A/D conversion unit 72 then outputs thedigital image signal as the digital image data P0. In this manner, theimaging unit 71 and the A/D conversion unit 72 function as the imageinput means 1.

Thereafter, the image processing unit 73 carries out white balanceadjustment processing, gradation correction processing, densitycorrection processing, color correction processing, and sharpnessprocessing, and outputs corrected image data P1. In this manner, theimage processing unit 73 functions as the image correction means 2.

The image P1 is displayed on the LCD of the display unit 78. As a mannerof this display can be used display of thumbnail images as shown in FIG.3A. While operating the operation buttons of the operation unit 76, thephotographer selects and enlarges one of the images to be processed, andcarries out selection from a menu for further manual image correction ormanipulation. In the case where “Blur Correction” is selected at thisstage, the control unit 70 starts a program for blur direction judgmentand blur correction stored in the internal memory 79, and causes theimage processing unit 73 to carry out the blur direction judging processand the blur correcting process (see FIG. 4) using the mathematicalmodel M stored in advance in the internal memory 79, as has beendescribed above. Processed image data P2 are then output. In thismanner, the functions of the image manipulation means 3 are realized.

The compression/decompression unit 74 carries out compression processingon the image data P2 according to a compression format such as JPEG, andthe compressed image data are written via the media recording unit 77 ina memory card inserted in the digital camera. In this manner, thefunctions of the image output means 4 are realized.

By installing the blur direction judging process and the blur correctingprocess of the present invention as the image processing function of thedigital camera, the same effect as in the case of the digital photographprinter can be obtained.

The manual correction and manipulation may be carried out on the imagehaving been stored in the memory card. More specifically, thecompression/decompression unit 74 decompresses the image data stored inthe memory card, and the image after the decompression is displayed onthe LCD of the display unit 78. The photographer selects desired imageprocessing as has been described above, and the image processing unit 73carries out the selected image processing.

Furthermore, the mathematical models for respective properties ofsubjects described by FIG. 12 may be installed in the digital camera. Inaddition, the processing for generating the mathematical model describedby FIG. 5 may be installed therein. A person as a subject of photographyis often fixed to some degree for each digital camera. Therefore, if themathematical model is generated for the face of each individual as afrequent subject of photography with the digital camera, the model canbe generated without variation of individual difference in face.Consequently, the blur direction judging process and the blur correctingprocess can be carried out with extremely high accuracy for an imageincluding the face of the person.

A program for causing a personal computer or the like to carry out theblur direction judging process and the blur correcting process of thepresent invention may be incorporated with image editing software. Inthis manner, a user can use the blur direction judging process and theblur correcting process of the present invention as an option of imageediting and manipulation on his/her personal computer, by installationof the software from a recording medium such as a CD-ROM to the personalcomputer, or by installation of the software through downloading of thesoftware from a predetermined Web site on the Internet.

1. A blur direction judgment method comprising the steps of: obtainingone or more weighting parameters for one or more statisticalcharacteristic quantities representing a direction of blur in apredetermined structure in an input image by fitting a modelrepresenting the structure to the structure in the input image, themodel having been obtained by carrying out predetermined statisticalprocessing on a plurality of images representing the predeterminedstructure in different conditions of blur, and the model representingthe structure by one or more statistical characteristic quantitiesincluding the statistical characteristic quantity or quantitiesrepresenting the direction of blur and by one or more weightingparameters for weighting the statistical characteristic quantity orquantities according to an individual characteristic of the structure;and judging the direction of blur represented in the input imageaccording to a value or values of the weighting parameter or parametershaving been obtained.
 2. A blur direction judgment apparatus comprising:parameter acquisition means for obtaining one or more weightingparameters for one or more statistical characteristic quantitiesrepresenting a direction of blur in a predetermined structure in aninput image by fitting a model representing the structure to thestructure in the input image, the model having been obtained by carryingout predetermined statistical processing on a plurality of imagesrepresenting the predetermined structure in different conditions ofblur, and the model representing the structure by one or morestatistical characteristic quantities including the statisticalcharacteristic quantity or quantities representing the direction of blurand by one or more weighting parameters for weighting the statisticalcharacteristic quantity or quantities according to an individualcharacteristic of the structure; and blur direction judgment means forjudging the direction of blur represented in the input image accordingto a value or values of the weighting parameter or parameters havingbeen obtained by the parameter acquisition means.
 3. The blur directionjudgment apparatus according to claim 2 further comprising blur widthacquisition means for finding a width of blur in the direction of blurhaving been judged by the blur direction judgment means.
 4. The blurdirection judgment apparatus according to claim 3, wherein the blurwidth acquisition means finds the width of blur based on the value orvalues of the weighting parameter or parameters for the statisticalcharacteristic quantity or quantities representing the direction ofblur.
 5. The blur direction judgment apparatus according to claim 4further comprising blur correction means for correcting the blurrepresented in the input image based on the direction of blur havingbeen judged by the blur direction judgment means and the width of blurobtained by the blur width acquisition means.
 6. The blur directionjudgment apparatus according to claim 5, wherein the blur correctionmeans corrects the blur in the structure in the input image by changingthe value or values of the weighting parameter or parameters for thestatistical characteristic quantity or quantities representing thedirection of blur.
 7. The blur direction judgment apparatus according toclaim 2 further comprising detection means for detecting the structurein the input image, wherein the parameter acquisition means obtains theweighting parameter or parameters by fitting the model to the structurehaving been detected.
 8. The blur direction judgment apparatus accordingto claim 2 further comprising selection means for obtaining at least oneproperty of the structure in the input image and for selecting the modelcorresponding to the obtained property from a plurality of the modelsrepresenting the structure for respective properties of thepredetermined structure, wherein the parameter acquisition means obtainsthe weighting parameter or parameters by fitting the selected model tothe structure in the input image.
 9. The blur direction judgmentapparatus according to claim 2, wherein the predetermined structure is ahuman face.
 10. The blur direction judgment apparatus according to claim2, wherein the model and the fitting of the model are realized by amethod of Active Appearance Model.
 11. A computer-readable recordingmedium storing a blur direction judgment program causing a computer tofunction as: parameter acquisition means for obtaining one or moreweighting parameters for one or more statistical characteristicquantities representing a direction of blur in a predetermined structurein an input image by fitting a model representing the structure to thestructure in the input image, the model having been obtained by carryingout predetermined statistical processing on a plurality of imagesrepresenting the predetermined structure in different conditions ofblur, and the model representing the structure by one or morestatistical characteristic quantities including the statisticalcharacteristic quantity or quantities representing the direction of blurand by one or more weighting parameters for weighting the statisticalcharacteristic quantity or quantities according to an individualcharacteristic of the structure; and blur direction judgment means forjudging the direction of blur represented in the input image accordingto a value or values of the weighting parameter or parameters havingbeen obtained by the parameter acquisition means.